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Reconstruction  of  Binocular  Depth  across  Continuous  Surfaces 


This  technical  report  consists  of  two  publications.  The  first  will  appear  in  the  Journal  of 
Experimental  Psychology:  Human  Perception  and  Performance  in  1989.  This  article 
examines  the  perception  of  depth  in  random  dot  surface  versus  volume  stimuli,  and  shows 
that  depth  associated  with  a  continuous  surface  is  subject  to  reconstruction  artifacts,  where 
equivalent  random-dot  volume  stimuli  are  used  as  a  control.  The  second  article  has  been 
submitted  for  publication.  This  article  explores  the  analogy  between  the  reconstruction  of 
stereoscopic  depth  and  the  reconstruction  of  brightness.  The  analogy  holds  for  a  number 
of  classical  simultaneous  contrast  effects.  Dissimilarities  are  found,  however,  in  terms  of 
the  lateral  inhibition  effects  traditionally  attributed  to  underlying  spatial-differention 
operators. 
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Abstract.  The  depth  of  a  pair  of  binocular  targets  embedded  in  surfaces  is  shown  to  be 
influenced  by  the  apparent  depth  and  slant  of  the  underlying  surfaces.  Random  dot 
stereograms  were  generated  with  a  sawtooth  (triangle-wave)  disparity  profile,  which 
corresponded  to  a  set  of  slanted  planar  surfaces  separated  by  sharp  depth  discontinuities. 
The  stimuli  were  chosen  to  capitalize  on  the  known  difficulty  in  perceiving  slanted  planes 
in  binocular  depth.  Two  targets  were  embedded  in  these  surfaces,  either  on  adjacent 
surfaces  separated  by  one  depth  edge  or  at  a  greater  separation  with  three  intervening 
depth  discontinuities.  The  targets  were  also  embedded  in  a  random  volume  of  points  as 
a  control  context.  The  apparent  depth  of  the  targets  was  influenced  by  the  context, 
with  the  surface  stimuli  misperceived  as  having  a  stairstep  profile  in  depth.  The  effect 
suggests  a  distinction  between  the  depth  processing  of  isolated  points  within  a  volume 
versus  points  associated  with  the  interiors  of  continuous  surfaces. 
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Introduction 

For  distances  measured  radially  from  an  observer,  the  depth  associated  with  a  given  loca¬ 
tion  is  the  difference  in  distance  between  that  location  and  a  given  reference  location. 
Depth,  which  is  generally  small  compared  to  the  overall  reference  distance,  is  often  used 
to  describe  incremental  distance  variations,  such  as  surface  relief.  Apparent  depth  is 
presumably  the  direct  perceptual  counterpart  to  this  geometric  quantity,  so  that  the 
apparent  three-dimensionality  of  viewed  surfaces  is  usually  expected  to  correspond  to  the 
determination  of  apparent  depth  for  points  across  the  given  surface.  There  is,  in  princi¬ 
ple,  a  direct  geometric  relationship  between  depth  and  binocular  disparity,  where  the 
point  of  convergence  of  the  two  eyes  provides  a  natural  reference  distance  (see  formula¬ 
tions  in  Foley,  1980;  Mayhew,  1982).  At  least  in  the  near  field,  depth  has  been  shown  to 
be  directly  related  to  disparity  and  convergence  (Richards  &  Miller,  1969;  Ritter,  1977, 
1979;  Foley,  1980;  Morrison  &  Whiteside,  1984).  The  visual  system  partially  compen¬ 
sates  for  the  dependency  of  depth  on  the  square  of  the  distance  to  the  point  of  conver¬ 
gence  (Ono  &  Comerford,  1977;  Wallach,  Gillam  &  Cardillo,  1979).  Foley  (1980)  has 
shown  that  systematic  errors  in  binocular  depth  can  be  attributed  to  errors  in  the  esti¬ 
mation  of  the  apparent  reference  distance  based  on  an  extraretinal  convergence  signal. 
Vertical  disparities  or  eye  movements  have  also  been  proposed  as  contributing  to  deter¬ 
mining  the  geometric  parameters  of  the  binocular  system  necessary  for  recovering  depth 
(Longuet-Higgins,  1982a,  b;  Mayhew,  1982;  Prazdny,  1983).  It  should  be  noted  that 
while  binocular  disparity  is  often  described  in  terms  of  absolute  retinal  positions,  there  is 
evidence  that  the  effective  binocular  disparity  is  determined  by  differences  between  the 
two  half-images,  as  suggested  by  our  ability  to  maintain  stable  fusion  despite  retinal 
motion  (Steinman  &  Collewijn,  1980;  Steinman,  Levinson,  Collewijn,  &  van  der  Steen, 
1985;  Lappin,  1985). 
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It  has  been  widely  presumed  that  binocular  depth  across  continuous  surfaces  is  a 
straightforward  extension  of  that  associated  with  discrete  binocular  features.  A  continu¬ 
ous  surface  would  present  a  rather  dense  sampling  of  binocular  features,  each  contribut¬ 
ing  to  the  impression  of  depth  at  the  corresponding  surface  location,  probably  on  the 
basis  of  local  disparity  differences  or  contrast  (Gogel,  1956,  1972;  Gulick  &  Lawson, 
1976).  The  importance  of  disparity  contrast,  and  not  absolute  disparity,  in  determining 
apparent  depth  was  first  suggested  by  certain  “depth  contrast”  effects  (Werner,  1938, 
1942;  Pastore,  1964;  Pastore  &  Terwilliger,  1966).  A  simple  example  of  depth  contrast  is 
that  of  a  central  line  at  0*  disparity  surrounded  by  flanking  lines  or  dots  that  have 
disparities  consistent  with  lying  on  a  slanted  plane:  the  central  line  will  appear  to  slant 
away  from  the  (apparently  unslanted)  frame.  Depth  contrast  effects  have  been  attri¬ 
buted  primarily  to  the  process  of  binocular  fusion,  e.g.  cyclotorsion  or  shifts  in  effective 
correspondence  (Ogle,  1946;  Nelson,  1977)  perhaps  with  the  apparent  frontoparallel,  or 
zero-disparity,  plane  influenced  by  monocular  cues  (Harker,  1962). 

While  disparity  contrast  seems  necessary  for  the  perception  of  apparent  depth, 
recent  observations  suggest  that  it  is  not  sufficient.  Specifically,  coplanar  arrangements 
of  binocular  features,  corresponding  to  slanted  planes,  have  been  found  relatively 
ineffective  in  inducing  apparent  slant.  Gillam,  Flagg,  and  Finlay  (1984)  found  that  the 
slant  of  a  plane  is  perceived  much  more  rapidly  when  bounded  by  disparity  discontinui¬ 
ties,  and  that,  in  their  absence,  depth  develops  with  a  slow  time  course  similar  to  that 
reported  in  “aniseikonia”  experiments  (Ames,  1946).  Mitchison  and  Westheimer  (1984) 
also  found  that  depth  derives  less  effectively  when  the  disparity  features  correspond  to  a 
coplanar  arrangement,  i.e.  lying  on  a  slanted  plane.  They  found  that  the  threshold  for 
detection  of  apparent  depth  is  elevated  when  adjacent  binocular  features  are  coplanar, 
and  that  the  slant  is  particularly  difficult  to  discern  for  certain  arrangements, 
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particularly  those  that  monocularly  suggest  an  unslanted  configuration,  such  as  a  square 
(Werner,  1937;  Westheimer,  1979;  McKee,  1983).  The  dominance  of  the  monocular 
interpretation  over  constant  disparity  gradients  was  shown  recently  (Stevens  &  Brookes, 
1988)  for  a  variety  of  stereograms  in  which  the  distribution  of  binocular  disparities 
corresponded  to  a  slanted  plane  whose  orientation  was  inconsistent  with  the  monocular 
interpretation  (e.g.  as  suggested  by  linear  perspective).  Given  a  sufficiently  compelling 
monocular  configuration,  even  very  large  contradictory  disparity  gradients  are  ineffective, 
provided  they  correspond  to  coplanar  binocular  features,  and  are  presented  in  the 
absence  of  boundary  disparity  contrast. 

The  observations  that  binocular  depth  is  dependent  on  the  presence  of  disparity 
contrast  and  that  depth  is  least  reliably  recovered  from  constant  disparity  gradients  sug¬ 
gests  an  analogy  between  depth  from  disparity  contrast  and  brightness  from  luminance 
contrast.  Central  to  the  analogy  is  that  binocular  depth,  like  brightness,  appears  to  be 
reconstructed  across  continuous  regions  bounded  by  contrast  edges,  as  demonstrated  by 
the  depth  analogue  of  the  Craik-O’Brien-Cornsweet  (Anstis,  Howard  &  Rogers,  1978). 
Other  brightness  analogues  can  be  demonstrated,  e.g.  a  constant  disparity  gradient 
induces  a  complementary  slant  in  a  ring  of  constant  disparity  (Stevens  &  Brookes,  1987; 
Stevens,  Note  1)  —  an  effect  likely  related  to  depth  induction  first  observed  by  Werner 
(1938).  See  (Brookes  &  Stevens,  1988)  for  discussion  of  the  limits  of  this  analogy. 

Several  explanations  have  been  offered  for  the  observed  insensitivity  to  low  spatial 
frequency  variations  in  disparity,  including  spatial  lateral  inhibition  (Anstis  et  al.,  1978; 
Tyler,  1983)  and  local  processes  that  align  retinal  images  prior  to  binocular  fusion 
(Anderson  &  Van  Essen,  1987).  But  while  some  low  spatial  frequency  depth  information 
is  seemingly  lost  at  an  early  stage  of  binocular  processing,  the  various  depth  contrast 
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effects  just  mentioned  show  that  at  least  some  of  that  information  is  subsequently  recon¬ 
structed.  But  other  than  demonstrating  the  existence  of  binocular  depth  reconstruction, 
little  more  is  known  about  it. 

Mitchison  and  Westheimer  (1984)  characterize  depth  as  being  derived  from 
differences  in  local  disparity  contrast,  observing,  for  example,  that  lines  that  have  the 
same  disparity  difference  between  themselves  and  their  neighbors  appear  at  equal  depths. 
This  accounts  for  a  variety  of  phenomena  involving  coplanar  binocular  arrangements 
which  exhibit  little  apparent  depth,  but  their  explanation  seems  to  us  more  closely  tied 
to  the  local  detection  of  surface  curvature  or  discontinuity  features  based  on  disparity 
than  on  the  overall  reconstruction  of  depth.  More  generally,  our  experience  with  similar 
stimuli  has  been  that  features  embedded  in  continuous  surfaces  assume  the  apparent 
depth  of  the  immediately  underlying  surface,  which  might  consequently  cause  the 
features  to  appear  at  different  depths,  on  the  basis  of,  e.g.,  monocular  cues  (Stevens  & 
Brookes,  1988).  We  examined  here  whether  this  tendency  also  holds  for  purely  binocular 
stimuli. 

Experiment 

The  approach  was  to  use  two  types  of  random  dot  stereogram  (RDS)  stimuli:  one  in 
which  the  dots  are  given  systematically  varying  binocular  disparities  that  correspond  to  a 
triangle-wave  surface,  i.e.  a  series  of  linear  ramps  separated  by  sharp  disparity  discon¬ 
tinuities  (figure  la);  in  the  second  type  of  RDS  stimulus,  which  served  as  a  control,  the 
points  were  distributed  randomly  in  disparity  so  that  they  appeared  to  lie  scattered 
throughout  a  volume  of  space  (figure  lb).  The  stimuli  were  presented  with  no  visible 
disparity  contrast  with  the  margins  of  the  display;  the  only  contrast  was  within  the 
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RDS,  either  among  the  dots  of  the  volume  stimuli  or,  in  the  case  of  the  triangle-wave 
stimuli,  across  the  vertical  margins  between  adjacent  slanted  planes.  Of  particular 
importance  to  this  experiment  is  the  fact  that  the  impression  of  overall  depth  from  the 
triangle-wave  disparity  profile  is  incorrect.  Rather  than  appearing  as  a  series  of  slanted 
planes  at  a  common  mean  distance  from  the  observer,  their  slant  in  depth  is  underes¬ 
timated,  so  that  the  sharp  disparity  discontinuities  between  planes  induce  an  erroneous 
overall  increase  in  depth  across  the  pattern.  The  RDS  is  seen  in  depth  immediately  as  an 
arrangement  of  slightly  slanted  planes,  whose  apparent  overall  depth  variation  is  inter¬ 
mediate  between  a  triangle-wave  and  a  staircase  profile.  The  magnitude  of  the  staircase 
effect  is  at  least  as  large  as  that  observed  in  the  depth  analogue  to  the  Craik-O’Brien- 
Cornsweet  effect  (Anstis  et  al.  1978).  Given  these  two  stimuli,  subjects  were  asked  to 
compare  the  relative  depth  of  two  embedded  target  points  that  could  be  readily  discerned 
from  the  other  points  of  the  RDS. 

The  intention  of  this  experiment  is  to  demonstrate  a  dependence  of  binocular  depth 
on  the  presence  of  continuous  surfaces.  The  stimuli  are  intended  to  be  purely  binocular, 
as  afforded  by  random  dot  stereograms  containing  no  monocular  surface  features.  It  is 
conceivable  that  the  fused  stereogram  contains  residual  monocular  depth  or  slant  cues 
that  might  influence  the  results.  For  example,  dot  density  was  uniform  across  the  stereo¬ 
grams,  and  the  individual  dots  were  all  the  same  size  (slightly  less  than  one  arc  minute), 
both  of  which  indicating  a  stimulus  equidistant  from  the  observer,  contrary  to  the  bino¬ 
cular  interpretation.  These  influences,  if  measureable,  would  apply  equally  to  all  RDS 
stimuli,  and  presumably  serve  to  reduce  the  impression  of  varying  depth.  The  more 
important  effect  pursued  here  is  the  influence  of  surfaces  on  the  apparent  depth  of 
embedded  target  points. 


<  figure  1  (a  and  b)  about  here  > 
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Method 

Apparatus:  The  RDS  stimuli  were  generated  by  a  Symbolics  3675  Lisp  Machine  and 
displayed  on  a  Wheatstone-style  stereoscope  consisting  of  a  pair  of  optically-flat  front- 
surfaced  mirrors  and  Tektronix  634  monochrome  displays.  The  monitors  were  94  cm 
from  the  observer,  as  measured  along  the  optic  axis  from  eye  to  screen,  and  viewed  with 
a  convergence  angle  consistent  with  the  observation  distance.  The  stimulus  stereogram 
subtended  approximately  7  *  and  consisted  of  luminous  points  against  a  dark  back¬ 
ground;  the  stereoscope  was  viewed  in  darkness. 

Stimuli:  For  the  RDS  triangle-wave  surface  stimuli  consisted  of  2000  points  whose  dispar¬ 
ities  corresponded  to  four  slanted  planes  each  subtending  1.8*  horizontally  by  5.8*  verti¬ 
cally.  Disparity  varied  linearly  across  each  slanted  plane  and  discontinuously  across  the 
vertical  margins  between  adjacent  planes.  The  overall  disparity  range  was  -1.53'  to 
6.13',  well  within  Panum’s  fusional  limit.  The  disparity  gradient  across  each  plane 
corresponded  to  one  of  two  slants  in  depth,  varying  either  4.6'  or  6.1'  over  the  1.8* 
width  of  the  plane  (see  disparity  profile  in  figure  2). 

<  figure  2  about  here  > 

Superimposed  onto  the  RDS  were  two  target  points,  each  subtending  3'  so  as  to  be 
distinguishable  from  the  RDS  points,  with  binocular  disparities  that  matched  the 
triangle-wave  profile  at  its  projected  location  so  that  each  target  appeared  to  lie  flush 
with  the  surrounding  RDS  surface.  The  two  targets  were  positioned  on  the  horizontal 
meridian  to  the  left  and  right  of  the  vertical  meridian.  The  targets  were  either  embedded 
in  the  central  two  planes  (separated  by  2  *  and  one  depth  edge)  or  the  outer  two  planes 
(separated  by  6  *  and  three  intervening  depth  edges).  We  will  refer  to  these  as  the  near- 
and  far-separation  conditions.  Figure  1  shows  the  two  targets  in  the  far-separation 
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condition.  For  each  of  the  two  separations,  the  targets  could  appear  in  slightly  different 
lateral  positions  on  their  corresponding  slanted  planes,  so  that  four  different  relative 
disparities  would  result,  specifically  ±1.5'  and  ±3.1'.  Geometrically,  a  positive  disparity 
difference  corresponded  to  the  left  target  nearer  than  the  right  while  a  negative  disparity 
difference  corresponded  to  the  right  target  nearer.  The  smaller  disparity  difference 
(±1.5')  was  chosen  empirically  to  be  a  challenging  relative  depth  task  for  targets 
separated  by  6  *  amidst  the  other  “distractor”  points  of  the  RDS.  Altogether  6  combi¬ 
nations  of  binocular  disparity  were  provided:  the  four  combinations  just  described  plus 
two  conditions  where  the  two  targets  had  equal  disparity  but  different  locations  on  their 
respective  surfaces.  Note  that  since  the  two  targets  appeared  on  ramps  of  differing 
disparity  gradient,  the  relative  depth  judgement  could  not  be  deduced  merely  from  their 
relative  placement  on  the  underlying  planes.  The  mean  disparities  of  the  ramps  were 
chosen  in  order  to  accommodate  the  range  of  relative  target  disparities. 

A  second  experimental  series  was  performed  with  the  two  targets  embedded  in  ran¬ 
dom  dot  volume  stimuli  (figure  lb).  In  this  case  the  same  2000  dots  were  given  random 
disparities  within  the  same  overall  disparity  range  of  -1.53'  to  6.13'  used  before.  The 
dots  that  comprised  the  volume  stimuli  were  fused  readily  and  immediately  appeared  to 
define  a  volume  of  distinct  points  that  were  scattered  in  depth.  The  same  six  combina¬ 
tions  of  target  location  were  used  in  these  volume  stimuli  in  conjunction  with  the  two 
disparity  senses  (normal  and  reversed).  Unlike  the  triangle-wave  surface  stimuli,  where 
the  targets  appeared  to  lie  on  surfaces  in  depth,  the  targets  in  this  series  appeared  to 
float  in  space  amidst  a  random  field  of  other  3D  points. 

Procedure:  Five  experienced  subjects  participated  in  the  experiment,  four  of  whom  were 
naive  to  the  nature  of  the  experiment;  all  had  good  stereo  vision.  In  each  trial  the 
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stimulus  RDS  was  presented  for  1000  msec  without  the  two  targets,  followed  by  an  addi¬ 
tional  750  msec  during  which  the  target  points  were  superimposed  on  the  RDS.  The 
subjects  were  told  that  they  would  see  a  pair  of  target  points  embedded  in  either  a 
configuration  of  surfaces  or  a  volume  of  points,  and  that  they  were  to  decide  quickly  but 
reliably  which  of  the  two  targets  appeared  closer  to  the  subject.  The  subject  indicated 
the  left  or  the  right  target  by  pressing  the  corresponding  button  on  a  mouse.  The  sub¬ 
jects  were  not  given  feedback  about  the  accuracy  of  their  judgments. 

The  experiment  consisted  of  two  series  of  trials,  the  triangle-wave  surface  stimuli 
(figure  la)  followed  by  the  random  volume  stimuli  (figure  lb).  Each  series  consisted  of 
120  trials  presented  in  random  order:  5  repetitions  of  each  of  12  distinct  stimulus  condi¬ 
tions,  each  presented  for  two  choices  of  RDS  disparity  sense  (normal  and  reversed,  which 
served  to  reverse  the  direction  in  which  depth  increased  in  the  apparent  staircase).  Note 
that  the  disparity  reversal  was  for  the  entire  stereogram,  including  the  targets.  The  12 
conditions  were  comprised  of  6  choices  of  position  for  the  two  targets  and  two  target 
separations  (near  and  far).  Subjects  were  given  learning  trials  without  feedback  until 
they  indicated  that  they  were  comfortable  with  the  task. 

Results  and  Discussion 

For  the  volume  stimuli,  the  relative  depth  of  the  two  targets  was  judged  reliably  for  both 
the  near  (2°)  and  far  (6')  separations.  The  far-separation  case,  not  surprisingly,  pro¬ 
duced  slightly  more  errors,  particularly  when  the  targets  differed  by  only  *±1.5'  in  dispar¬ 
ity  (17%  errors  of  the  corresponding  trials).  In  comparison,  when  the  targets  differed  by 
±3.1 '  in  disparity,  their  relative  depth  was  judged  accurately  (1%  error)  despite  the  large 
separation  and  the  many  intervening  depth  points. 
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Th  -e  performance  was  quite  different  when  the  target  points  were  embedded  in  the 
triangle-wave  stimulus.  An  ANOVA  was  performed  to  test  the  main  effects  of  i)  the  sur¬ 
face  versus  volume  background,  ii)  target  separation,  and  in)  disparity  difference.  The 
presence  of  the  surface  was  found  to  be  significant  (F(l,4)  =  22.84,  p  <  .05).  For  the 
far-separation  case,  subjects  had  a  strong  tendency  to  make  relative  depth  judgments 
consistent  with  the  targets  lying  on  separate  planes  in  depth  arranged  as  a  staircase 
(rather  than  as  a  triangle-wave  profile  of  slanted  planes),  despite  the  contradictory  depth 
ordering  implied  by  their  disparities.  For  targets  that  were  separated  by  only  2°  (and 
lying  on  adjacent  planes)  the  depth  judgments  were  more  in  accordance  with  disparity, 
but  still  were  judged  contrary  to  disparity  in  22%  of  the  trials,  in  comparison  to  5%  for 
the  volume  stimuli.  This  corresponds  to  the  subjective  impression  that  the  illusory  stair¬ 
case  is  relatively  weak  over  adjacent  step  discontinuities,  and  is  most  apparent  when 
judging  the  relative  depth  of  two  points  separated  by  several  step  edges.  The  cases  most 
consistent  with  the  illusory  staircase  involved  targets  separated  by  6  °  (three  intervening 
step  discontinuities)  and  1.5'  in  disparity:  the  targets  were  seen  in  depth  according  to 
the  apparent  staircase  and  contrary  to  their  disparity  difference  in  90%  of  the  trials. 
Even  for  targets  with  a  disparity  difference  of  3.1',  their  relative  depth  was  contrary  to 
disparity  in  66%  of  the  trials,  compared  to  1%  in  the  corresponding  volume  stimuli. 

In  table  1  the  data  are  collapsed  across  disparity  reversals  and  presented  in  a 
manner  that  emphasizes  the  degree  to  which  the  responses  were  consistent  with  the  stair¬ 
case  depth  interpretation.  As  a  basis  for  comparison,  the  first  row  shows  how  relative 
depth  would  be  judged  if  based  exclusively  on  binocular  disparity.  The  data  are 
presented  with  the  convention  that  the  apparent  staircase  increased  in  depth  from  left  to 
right,  so  that  a  positive  disparity  difference  would  be  consistent  with  the  staircase.  Note 
that  the  two  conditions  where  the  targets  had  0°  disparity  difference  are  presented 
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together  in  the  center  column.  For  the  volume  stimuli  there  would  be  no  expected  bias 
(hence  the  .5  prediction)  but  for  the  surface  stimuli  we  expected  a  bias  if  the  targets  were 
seen  as  lying  at  different  depths  on  the  apparent  staircase.  Note  that  the  data  for  sur¬ 
face  stimuli  show  an  orderly  bias  towards  the  staircase  interpretation,  particularly  for  the 
far-separation  case. 

<  table  1  about  here  > 

It  has  been  demonstrated  by  many  studies  that  depth  can  be  derived  reliably  from 
disparity  contrast  for  spatially  isolated  targets.  The  targets  in  these  stimuli  were  not 
isolated,  however.  It  has  been  shown  that  binocular  points  in  close  proximity,  separated 
by  less  than  about  6-8',  exhibit  depth  averaging  and  spatial  attraction  and  repulsion 
effects  (Mitchison  &  McKee,  1985;  Westheimer,  1986;  Westheimer  &  Levi,  1987).  In  our 
experiment  the  dot  density  was  such  that  several  background  points  could  be  expected  to 
lie  within  approximately  6'  of  each  target.  It  is  therefore  conceivable  that  the  relative 
depth  of  the  targets  was  perturbed  by  adjacent  RDS  points,  which  contributed  to  the 
observed  error  rates.  These  perturbations,  of  course,  would  not  have  systematically 
influenced  the  target  depths  in  the  triangle- wave  stimuli. 

Probably  more  relevant  is  a  second  type  of  spatial  interaction,  which,  as  discussed 
earlier,  tends  to  reduce  apparent  depth  among  coplanar  binocular  features.  We  expect 
that  binocular  depth  is  reconstructed  across  regions  of  continuous  disparity  change  on 
the  basis  of  the  boundary  conditions,  such  as  the  sharp  disparity  discontinuities  between 
adjacent  planes  in  the  triangle-wave  stimuli.  Since  we  are  relatively  insensitive  to  the 
disparity  gradient  within  the  individual  slanted  planes,  the  reconstruction  process  errone¬ 
ously  accumulates  an  overall  depth  increase  across  subsequent  planes,  giving  the  impres¬ 
sion  of  a  staircase  in  depth.  Since  the  targets  were  perceived  as  lying  on  the  surfaces, 
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their  apparent  depths  were  consequently  subject  to  errors  of  depth  reconstruction. 

Conclusions 

The  relative  depth  of  a  pair  of  isolated  targets  separated  by  a  visual  angle  of  several 
degrees  and  by  several  minutes  of  disparity  can  be  readily  determined  from  their  binocu¬ 
lar  disparities.  In  the  present  experiment  this  was  demonstrated  by  the  volume  condition 
in  which  the  two  targets  were  embedded  in  a  volume  of  distractor  points.  In  the  surface 
condition,  the  positions  of  the  distractor  points  were  held  constant,  but  their  disparities 
were  distributed  systematically  rather  than  randomly  in  depth.  Thus,  the  only  difference 
between  stimuli  in  the  two  conditions  was  in  the  disparity  distribution  of  the  distractor 
points.  When  the  distractor  points  defined  continuous  surfaces,  the  relative  depths  of 
the  embedded  targets  were  no  longer  determined  solely  by  their  relative  disparities. 
Rather,  the  target  points  acquired  the  depth  of  the  embedding  surfaces,  and  thus  became 
subject  to  reconstructive  errors  in  the  perception  of  the  surfaces. 

The  surface  condition  stimuli  were  designed  to  introduce  an  illusory  impression  of  a 
staircase  in  depth,  by  capitalizing  on  the  relatively  greater  percieved  depth  produced  by 
sharp  disparity  edges  than  by  continuous  ramps  of  similar  overall  disparity  contrast. 
This  effect  allowed  us  to  demonstrate  that  depth  judgements  for  target  points  on  con¬ 
tinuous  surfaces  are  mediated  by  processes  that  access  the  reconstructed  depth  of  the 
underlying  surfaces,  rather  than  being  determined  by  their  true  disparity  difference. 
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Table  1. 

Fraction  of  Depth  Responses  as  a  Function  of  Disparity  Difference 


Disparity  difference 


-3.1 

-1.5 

0.0;0.0 

1.5 

3.1 

Predicted  fraction 

0.0 

0.0 

.50;.50 

1.0 

1.0 

Volume 

Near-separation 

.02 

.08 

.36;. 56 

.96 

.98 

Far-separation 

.02 

.14 

.40;.42 

.80 

1.0 

Surface 

Near-separation 

.10 

.34 

.88;. 64 

.98 

.98 

Far-separation 

.66 

.90 

.96;. 90 

.98 

.98 

Note.  Depth  responses  as  a  function  of  the  difference  in  disparity  of  the  two  target 
points,  for  combinations  of  target  separation  and  surface  versus  volume.  The  disparity 
differences  are  indicated  in  arc  minutes.  The  central  column  shows  the  two  conditions 
where  the  targets  had  equal  disparity.  The  first  row  shows  the  fraction  of  depth  judge¬ 
ments  predicted  purely  on  the  basis  of  their  relative  binocular  disparities,  hence  0.5  for 
the  two  cases  of  equal  disparity.  Below  are  the  fraction  of  judgments  consistent  with  the 
illusory  staircase  in  depth  seen  in  the  sawtooth  surface  stimuli,  where  1.0  would  indicate 
all  depth  judgments  corresponding  with  the  targets  lying  on  separate  levels  of  the 


apparent  staircase. 
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Figure  1.  RDS  stimuli  similar  to  those  used  in  the  experiment.  In  a)  the  disparities 

correspond  to  a  triangle-tvave  surface,  but  seen  as  having  an  overall  staircase  variation  in 

depth.  In  i)  the  disparities  are  distributed  randomly,  giving  the  appearance  of  a  volume 
of  points. 
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relative  disparities.  Crossed  disparities  are  negative  in  this  figure. 
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Abstract.  Apparent  depth  in  stereograms  exhibits  various  simultaneous-contrast  and 
induction  effects  analogous  to  those  reported  in  the  luminance  domain.  This  behavior 
suggests  that  stereo  depth,  like  brightness,  is  reconstructed  i.e.  recovered  from  higher- 
order  spatial  derivatives  or  differences  of  the  original  signal.  We  examine  the  extent  to 
which  depth  is  analogous  to  brightness,  and  find  similarities  in  terms  of  contrast  effects  but 
dissimilarities  in  terms  of  the  lateral  inhibition  effects  traditionally  attributed  to  underlying 
spatial-differentiation  operators. 

1  Introduction 

Stereo  disparity  contrast  can  induce  "depth  contrast"  in  a  manner  analogous  to  various  well- 
known  brightness  contrast  effects.  A  classic  brightness  contrast  demonstration  is  shown  in 
figure  la,  a  variant  of  Koffka's  ring  (Koffka  1935).  A  uniform-luminance  circle  is 
embedded  in  a  constant  luminance  gradient  background.  The  variable  contrast  between  the 
ring  and  its  immediate  background  induces  variable  apparent  brightness  around  the  circle. 
Analogously,  the  stereogram  in  figure  lb  consists  of  a  uniform-disparity  circle  embedded 
in  a  constant  disparity  gradient  background.  The  circle  appears  slanted  in  depth  in  the 
direction  opposite  that  of  the  background  gradient.  Just  as  the  brightness  in  figure  la  is 
dependent  on  luminance  contrast  more  than  absolute  luminance,  so  is  apparent  depth  in 
figure  lb  dependent  more  on  disparity  contrast  than  on  absolute  disparity. 

Depth  contrast  effects  were  first  observed  in  simple  stereograms  in  which  a  figure  at 
zero  disparity  appears  to  slam  in  depth  as  a  consequence  of  its  surrounding  context  (Werner 
1938,  1942;  Pastore  1964;  Pastore  and  Terwilliger  1966).  Ogle  (1946)  suggested  that 
during  the  fusion  process,  in  attempting  to  bring  the  context  to  zero  disparity,  cyclotorsion 
induces  opposite  disparity  in  the  figure.  Nelson  (1977)  later  provided  various  experiments 
that  ruled  out  cyclotorsion  as  the  sole  explanation,  and  furthered  Werner's  (1938)  proposal 
that  disparity  contrast  is  responsible  for  the  induction  of  apparent  depth.  In  a  manner 
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analogous  to  the  relationship  between  brightness  and  luminance  contrast,  the  apparent 
depth  in  certain  stereograms  seems  more  reliably  related  to  disparity  contrast  than  to 
absolute  disparities. 

The  analogy  between  depth  and  brightness  was  made  explicit  in  discussion  of  a 
stereoscopic  counterpart  of  the  Craik-O'Brien-Comsweet  illusion  (Anstis,  Howard  and 
Rogers  1978;  Rogers  and  Graham  1983).  In  the  luminance  version  of  this  illusion,  two 
fields  of  equal  luminance  meet  at  a  border  whose  profile  is  shaped  like  a  double  spur.  The 
impression  is  of  two  homogeneous  regions  differing  in  brightness  separated  by  a  sharp 
step  edge.  In  the  depth  version,  one  of  the  fields  is  seen  as  closer.  The  illusion 
demonstrates  that  depth  information  is  extrapolated  over  extended  regions  bounded  by 
sharp  disparity  edges,  much  like  the  extrapolation  of  brightness  information  away  from 
intensity  edges. 


<  figure  1  (a  and  b)  about  here  > 

Brightness  perception  has  been  treated  mathematically  as  two-dimensional 
integration  of  a  derivative- like  retinal  signal  (Arend  1973;  Blake  1985;  Schiffman  and 
Crovitz  1972;  Arend  and  Goldstein  1987).  Brightness  artifacts  can  be  regarded  as  failures 
to  achieve  an  accurate  reconstruction  of  the  incident  signal,  in  part  due  to  information  lost 
by  the  initial  derivative-like  measurements  (e.g.  from  thresholding). 

Several  brightness  phenomena  can  be  neatly  described  in  terms  of  an  empirically- 
measured  spatial  modulation  transfer  function  (MTF)  (Comsweet  1970).  The  retinal 
receptive  field  presumed  to  be  largely  responsible  for  the  overall  shape  of  the  MTF  is 
traditionally  modelled  as  a  difference  of  Gaussians  (DOG),  (Rodiek  and  Stone  1965; 
Enroth-Cugell  and  Robson  1966).  The  resemblance  of  this  circular- symmetric  operator  to 
the  Laplacian  of  a  Gaussian  has  been  noted  (Marr  and  Hildreth  1980),  although  the  actual 
ratio  of  space  constants  (between  excitatory  and  inhibitory  Gaussians)  in  retinal  DOGs  is 
far  too  great  to  constitute  a  quantitative  approximation  to  the  Laplacian  of  a  Gaussian 
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(Robson  1983).  Nonetheless,  center-surround  antagonism  provides  the  qualitative  effect 
of  Laplacian  filtering,  and  the  component  Gaussian  receptive  fields  of  the  DOG  achieves 
the  effect  of  low-pass  filtering,  relative  to  the  size  of  the  operator.  Lateral  inhibition  thus 
underlies  both  the  insensitivity  to  low  spatial  frequency  luminance  variations  and  for  the 
relative  sharpening  of  sensitivity  to  luminance  discontinuities  (both  of  which  are 
demonstrated  by  the  Craik-O'Brien-Comsweet  illusion).  Lateral  inhibition  has  also  been 
invoked  to  explain  other  instances  of  diminished  sensitivity  to  low  spatial  frequencies,  e.g. 
of  line  spacing,  line  length,  and  velocity  (MacKay  1973;  Loomis  and  Nakayama  1973; 
Crovitz  1976). 

Stereo  depth  likewise  exhibits  an  effective  spatial  MTF.  Sensitivity  to  sinusoidal 
spatial  modulations  of  stereo  disparity  is  limited  to  about  a  maximum  of  5  cycles/deg,  with 
peak  sensitivity  at  about  1  cycle/deg,  and  gradually  diminishing  sensitivity  with  decreasing 
spatial  frequencies  (Tyler  1973, 1975).  The  maximum  sensitivity  and  the  high-frequency 
limits  are  consistent  with  independent  evidence  that  continuous  disparity  distributions  are 
spatially  integrated  within  areas  of  approximately  0.5  deg  in  diameter  (Tyler  and  Julesz 
1980).  The  gradual  low  frequency  fall-off  has  been  attributed  to  spatial  lateral  inhibition, 
e.g.  by  center-surround  antagonism  (Anstis  et  al.  1978;  Schumer  and  Ganz  1979;  Tyler 
1983;  Schumer  and  Julesz  1984).  It  should  be  noted  that  two  types  of  lateral  inhibition  can 
be  expected  in  disparity  processing:  i)  spatial  interactions,  with  summation  or  pooling  of 
disparity  signals  within  subfields  and  (center-surround)  antagonism  across  spatially- 
separated  subfields,  and  ii)  inhibition  across  disparity-tuned  channels  at  a  common  location 
(Richards  1972;  Tyler  and  Foley  1974;  Nelson  1975;  Julesz  1978;  Marr  and  Poggio  1980; 
Westheimer  1986;  cf  Prazdny  1985).  The  high  spatial  frequency  limit  would  be  evidence 
for  spatial  pooling  or  averaging  of  the  disparity  of  closely-spaced  features.  Recently, 
Westheimer  and  Levi  (1987)  showed  that,  within  about  4-6  arc  min,  binocular  points  show 
attraction  in  depth,  and  beyond  that  distance,  repulsion  in  depth. 
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Do  the  substantial  similarities  between  depth  contrast  and  brightness  contrast 
phenomena  reflect  similar  processing  strategies?  We  suggest  that  the  observed  similarities 
arise  primarily  from  the  fact  that  binocular  depth  and  brightness  are  both  reconstructed  from 
(disparity  or  luminance)  contrast,  but  that  the  analogy  is  limited  because  the  corresponding 
contrast  features  are  detected  by  fundamentally  different  strategies.  The  analogy  is  further 
limited  by  some  evidence  that  the  reconstruction  strategies  themselves  also  differ. 

The  discussion  that  follows  gives  instances  where  the  analogy  holds  dramatically 
and  obviously,  and  others  where  the  analogy  seems  to  fail.  Where  we  report  it  fails,  we 
are  summarizing  our  experience  over  a  variety  of  stimuli  using  several  experienced  stereo 
observers.  In  the  cases  where  the  analogy  holds,  the  effect  in  stereo  depth  is  similar  in 
strength  to  the  traditional  brightness  effect.  On  the  other  hand,  we  were  unable  to  find  a 
stereo  counterpart  for  several  other  brightness  effects.  The  breakdown  of  the  analogy  in 
these  instances  is  regarded  as  significant  in  light  of  the  strength  and  robustness  of  the 
original  brightness  effects. 

2  Brightness  and  Depth  Effects  Associated  with  Reconstruction 

The  Craik-O'Brien-Comsweet  illusion  in  stereo  depth  is  compelling  evidence  that  stereo 
depth  derives  from  a  process  that  reconstructs  surfaces  indirectly  from  boundary  contrast. 
There  are  other  demonstrations  that  depth  derives  from  relative  disparities,  i.e.  disparity 
differences  within  the  binocular  configuration,  as  opposed  to  absolute  retinal  disparities 
(Steinman  &  Collewijn  1980;  Lappin,  1985).  The  stereo  analogue  of  the  Craik-O'Brien- 
Comsweet  effect  further  shows  that  stereo  depth  is  subject  to  errors  in  integrating  overall 
depth  differences  from  subthreshold  disparity  variations.  The  difference  in  apparent 
distance  from  the  observer  to  the  left  and  right  extremes  of  the  pattern  reflects  a  failure  to 
incorporate  the  very  low  spatial  frequency  changes  into  the  accumulated  depth  variation 
over  the  pattern.  Note  that  judging  which  side  appears  closer  requires  comparing  apparent 
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radial  distances.  It  is  therefore  remarkable  that  even  with  free  eye  movements  observers 
cannot  perform  the  task  by  directly  comparing  the  disparities  of  the  two  regions.  Clearly 
the  distribution  of  the  surfaces  in  space  is  dominated  by  a  (disparity)  contrast-based 
reconstruction,  seemingly  in  close  analogy  to  the  reconstruction  of  brightness  in  the 
original  illusion. 

The  demonstration  of  "simultaneous  disparity  contrast"  in  figure  1  further  shows 
that  the  perception  of  depth  differences  and  of  slant  derives  from  local  disparity  contrast, 
e.g.  across  disparity  discontinuities.  Apparent  slant  across  a  continuous  surface  is  no 
more  reliably  related  to  the  local  disparity  gradient  than  is  absolute  depth  to  absolute 
disparity.  The  effect  is  thus  closely  analogous  to  brightness.  One  can  readily  generate 
further  depth  induction  counterparts  to  other  brightness  induction  demonstrations.  For 
instance,  just  as  two  adjacent  bars  of  the  same  luminance  have  different  apparent 
brightnesses  when  presented  against  a  luminance  ramp  background,  adjacent  lines  of  equal 
disparity  appear  at  different  depths  when  presented  with  a  uniform  disparity  gradient 
background  (Mitchison  and  Westheimer  1984).  These  effects  are  not  at  all  subtle:  the  ring 
in  figure  lb  appears  dramatically  slanted  despite  its  uniform  binocular  disparity. 

The  local  nature  of  the  depth  induction  effect  can  be  demonstrated  by  means  of  a 
nonlinear  background  gradient,  as  shown  in  figure  2.  In  the  luminance  version  (figure  2a) 
the  constant-luminance  ring  is  embedded  in  a  Gaussian-shaped  luminance  profile.  The 
brightness  of  the  ring  likewise  varies  with  opposite  sign  to  the  background  gradient  In  the 
corresponding  depth  version  the  constant-disparity  ring  is  embedded  in  a  Gaussian-shaped 
ridge  in  depth  (figure  2b).  The  ring  appears  to  curve  in  depth  with  induced  curvature 
opposite  to  that  of  the  background  ridge.  The  curvature  in  depth  induced  in  the  constant- 
disparity  ring  is  consistent  with  depth  being  dominated  by  the  local  disparity  contrast,  as  is 
brightness  by  local  luminance  contrast. 


<  figure  2  (a  and  b)  about  here  > 
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Simultaneous  brightness  contrast  is  also  seen  when  two  squares  of  equal  luminance 
are  embedded  in  backgrounds  of  differing  luminance.  The  square  in  the  lighter  background 
appears  darker  than  the  square  in  the  darker  background.  Does  it  have  a  counterpart  in 
stereo  depth?  The  corresponding  stereogram  (figure  3)  consists  of  two  squares  of  equal 
binocular  disparity  embedded  in  regions  of  opposite  disparity  sign.  For  the  analogy  to 
hold,  the  square  embedded  in  the  negative-disparity  background  should  appear  farther  than 
that  embedded  in  the  positive-disparity  background.  But  we  rind  no  corresponding  depth 
difference  in  this  configuration:  the  squares  appear  equidistant  from  the  observer.  The 
brightness-contrast  effect  is  often  attributed  to  a  logarithmic  transformation  of  incident 
luminance  (Comsweet  1970):  the  compressive  transformation  results  in  differing  effective 
contrasts  prior  to  lightness  reconstruction,  and  consequently  differing  apparent 
brightnesses.  But  no  corresponding  compressive  transformation  is  found  or  expected  for 
disparity. (see  Foley  and  Richards  1971;  Foley  1980). 

<  figure  3  about  here  > 

Another  simultaneous  contrast  effect  is  the  apparent  variation  in  brightness  within  a 
region  of  constant  luminance  induced  by  the  contrast  across  its  borders  with  adjacent 
regions.  Figure  4a,  for  example,  shows  a  staircase  pattern  of  rectangles  of  progressively 
higher  luminance  from  left  to  right.  Each  rectangle  appears  lighter  near  the  left  margin  and 
darker  toward  the  right,  an  effect  explicable  quite  directly  in  terms  of  the  spatial  MTF 
(Comsweet  1970). 

Figure  4  presents  the  analogous  stereo  stimulus:  a  staircase  disparity  profile.  The 
apparent  depth  profile  is  roughly  analogous  to  the  brightness  version:  the  individual 
rectangles,  despite  their  uniform  disparity,  appear  slanted  in  depth.  Although  the  depth 
increment  across  each  sharp  discontinuity  is  perceived  rather  accurately,  apparent  depth 
does  not  accumulate  correctly  over  the  staircase.  As  a  result,  the  overall  arrangement 
resembles  a  set  of  louvers,  with  the  left  side  of  each  slanted  rectangle  appearing  farther  that 
the  right. 
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<  figure  4  about  here  > 

The  misperception  of  depth  in  the  disparity  staircase  is  predicted  by  the  stereo  MTF, 
much  as  the  corresponding  contrast  sensitivity  MTF  predicts  apparent  brightness  for  the 
luminance  staircase.  But  more  is  involved  than  is  captured  merely  by  a  bandpass-filter 
model.  A  repeating  triangle-wave  disparity  pattern,  with  constant  mean  disparity  over  the 
pattern,  would  be  predicted  on  the  basis  of  the  MTF  to  be  seen  in  depth  veridically,  but  in 
fact  is  misperceived  as  a  staircase  depth  profile  (Brookes  and  Stevens  1989).  Apparent 
depth  increases  across  the  pattern  in  a  manner  analogous  to  the  accumulation  of  brightness 
reported  for  triangle-wave  luminance  profiles  sequences  of  Craik-O’Brien-Comsweet 
edges  (Arend,  Buehler  and  Lockhead,  1971;  Arend  1973;  but  see  Coren,  1983). 
Exceptions  to  the  analogy  concern  the  failure  to  observe  perturbations  to  the  apparent  depth 
profile  in  the  vicinity  of  disparity  discontinuities,  the  analogues  of  luminance  effects 
traditionally  attributed  to  lateral  inhibition.  We  discuss  this  aspect  of  the  analogy  next 

3  Effects  Associated  with  Lateral  Inhibition 

Several  brightness  phenomena  appear  to  directly  implicate  neural  mechanisms  that  might 
underlie  aspects  of  the  effective  spatial  MTF  of  the  visual  system.  The  first  such 
mechanisms  in  the  visual  pathway  are  the  retinal  ganglion  cells  which,  as  mentioned, 
perform  (spatiotemporal)  derivative-like  filtering  by  spatial  lateral  inhibition. 

Mach  bands  are  perhaps  the  most  compelling  illustration  of  lateral  inhibition.  The 
effect  is  an  apparent  creasing  of  the  brightness  profile  where  the  corresponding  luminance 
profile  exhibits  a  sharp  discontinuity  in  second  derivative.  For  example,  dark  and  light 
lines  are  seen  where  a  luminance  ramp  abuts  the  adjoining  dark  and  light  regions, 
respectively.  Mach's  proposal  that  the  phenomenon  derives  from  "reciprocal  action",  i.e. 
lateral  inhibition,  of  neighboring  areas  within  the  retina,  was  later  supported  by  direct 
neurophysiological  recordings  (Hartline  and  Ratliff  1965).  Mach  bands  are  robust  over  a 
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wide  range  of  luminance  gradients,  persist  under  focal  scrutiny,  and  have  measurable 
apparent  width  and  amplitude,  which  can  be  related  to  the  size  of  corresponding  center- 
surround  receptive  fields  in  the  retina  (Ratliff  1965). 

The  Hermann  grid  illusion  has  been  attributed  to  lateral  inhibition,  and  specifically 
to  center-surround  receptive  fields  (Baumgartner  1960).  The  illusory  spots  seen  at  the  grid 
intersections  are  consistent  with  the  expected  size  of  retinal  center- surround  receptive  fields 
(Ratliff  1965;  Spillman  1977).  It  should  be  noted  that  while  the  effect  is  likely  due  to 
lateral  inhibition,  it  is  doubtful  that  it  arises  solely  from  circular-symmetric  retinal  receptive 
fields;  orientation-selective  units  have  been  also  been  implicated  (Levine  et  al  1980;  Oehler 
and  Spillman  1981;  Wolfe  1984). 

Several  independent  results  would  suggest  that  the  lateral  inhibition-induced 
features,  if  present,  would  be  at  least  6  arc  min  wide.  Tyler  (1973)  showed  that  there  is  an 
upper  limit  of  about  5  cyc/deg  in  the  detection  of  sinusoidal  variations  in  depth  in 
stereograms,  which  is  equivalent  to  a  half-cycle  of  6  arc  min.  Mitchison  and  McKee 
(1987)  report  depth  averaging  for  dots  separated  by  less  than  about  6  arc  min.  Also, 
Westheimer  and  Levi  (1987)  demonstrate  a  transition  between  attraction  and  repulsion  in 
depth  for  targets  separated  by  about  4-6  arc  min.  The  attraction  and  repulsion  effect  is  not 
particularly  subtle:  the  magnitude  of  the  apparent  depth  purturbation  can  be  on  the  order  of 
a  minute  of  arc.  Thus,  if  the  spatial  processes  underlying  these  various  lateral  inhibition 
effects  were  to  induce  depth  analogues  to  the  corresponding  binocular  Hermann  grid  or 
Mach  band  stimuli,  they  should  occur  at  approximately  this  scale  and  magnitude,  or  larger 
parafoveally. 

In  the  depth  version  of  the  Hermann  grid,  consisting  of  a  grid  of  squares  above  a 
background  plane,  the  analogous  effect  would  be  illusory  depth  variations  in  the 
background  at  the  grid  intersections  (either  bumps  or  dips,  depending  on  the  disparity  of 
the  squares  relative  to  the  background  grid).  But  the  stereo  analogue  does  not  produce 
apparent  illusory  depth  distortions  at  the  grid  intersections  (Julesz  1965).  Figure  5  shows  a 
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representative  stereo  depth  version  of  the  Hermann  grid.  The  background  surface  appears 
uniformly  planar,  both  where  fixated  and  parafoveally. 

<  figure  5  about  here  > 

Figure  6  shows  the  stereo  analogue  to  the  ramp-like  luminance  profile  that  generates 
the  traditional  Mach  bands  in  brightness.  The  stereogram  consists  of  a  linear  disparity 
gradient  flanked  by  regions  of  uniform  disparity.  The  depth  analogue  to  a  Mach  band 
would  be  line-like  ridges  and  troughs  in  depth  where  the  disparity  ramp  abuts  the  negative- 
and  positive-disparity  regions,  respectively. 

<  figure  6  about  here  > 

In  examining  for  depth  Mach  bands,  we  used  both  dot  and  short-line  stimuli  with 
densities  similar  to  that  in  figure  6,  and  primarily  varied  the  slope  of  the  linear  ramp  region 
with  disparity  gradients  that  ranged  from  1:8  to  1:3.  For  the  moderately  shallow  1:8 
disparity  gradient,  the  disparity  varied  over  a  total  of  10  arc  min  across  the  length  of  the 
ramp.  The  spacing  between  adjacent  dots  or  short  lines  was  varied  over  a  range  of  2.3  arc 
min  to  6. 1  arc  min  with  increments  of  about  0.8  arc  min.  Also,  because  of  the  known 
anisotropy  between  horizontal  and  vertical  configurations  (Tyler  1973;  Wallach  and  Bacon 
1976;  Rogers  and  Graham  1983)  both  orientations  were  used  for  each  spacing.  No  Mach 
band-like  depth  effects  were  observed  in  stimuli  where  the  ramp  met  the  flanking  level 
regions  at  a  sharp  crease,  at  any  slope  or  orientation  of  stimulus.  However,  when  the 
disparity  profile  was  subtly  modified  to  mimic  Mach  bands  by  the  addition  of  slight  ridges 
and  troughs  (0.8  arc  min  amplitude)  at  the  margins  between  the  ramp  and  the  flanking 
regions,  observers  could  readily  discern  the  mock  Mach  bands. 

A  brightness  effect  similar  to  the  Mach  band  is  also  to  be  found  in  a  staircase 
luminance  profile  (e.g.  figure  4a).  In  the  immediate  vicinity  of  each  stairstep  the  brightness 
profile  appears  curved,  an  effect  attributed  to  lateral  inhibition  (Ratliff  1965;  Comsweet 
1970).  The  analogous  depth  effect  would  cause  the  uniform-disparity  rectangles  to  appear 
curved  as  well  as  slanted  in  depth.  But  while  the  rectangles  do  appear  slanted  (figure  4b), 
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they  appear  distinctly  planar.  The  disparity  contrast  across  the  step  edge  does  not  induce  a 
local  purturbation  to  the  apparent  surface  in  the  vicinity  of  the  edge. 

While  subde  depth  effects  analogous  to  Mach  bands  and  the  Herman  Grid  effect 
might  eventually  be  demonstrated,  we  find  it  noteworthy  that  the  analogous  lateral 
inhibition  effects  are  not  readily  apparent,  particulary  given  that  discrete  stereo  features 
have  been  shown  to  exhibit  substantial  depth  attraction  and  repulsion  when  brought  into 
close  proximity.  This  discrepancy  suggests  two  possibilities,  presuming  the  absence  of  the 
analogous  effects  is  valid.  Recall  that  Laplacian-like  filtering  enhances  luminance  changes 
and  facilitates  their  subsequent  localization,  and  that  Laplacian-like  filtering  can  be  achieved 
by  lateral-inhibition  or  center-surround  antagonism.  One  possibility,  then,  is  that  while 
some  binocular  mechanisms  incorporate  spatial  lateral  inhibition,  those  mechanisms  are  not 
involved  in  the  detection  of  disparity  change  (i.e.  depth  edges).  Alternatively,  lateral 
inhibition  artifacts  might  be  induced  in  depth  by  center-surround  disparity-summating 
mechanisms  but  later  suppressed  at  a  subsequent  stage  of  surface  perception.  These 
alternatives  are  discussed  further  below. 

4  General  Discussion 

The  main  points  of  the  analogy  between  stereo  depth  and  brightness  contrast  are  i)  both 
brightness  and  depth  appear  to  be  reconstructions  from  boundary  contrast  features  and  ii) 
both  luminance  and  disparity  contrast  features  are  seemingly  defined  by  discontinuities  or 
second  spatial  differences.  The  first  point  is  supported  by  a  range  of  contrast  effects  which 
establish  the  dependence  of  depth,  like  brightness,  on  the  available  boundary  conditions, 
several  of  which  were  shown  above.  The  second  point  is  supported  by  many  studies  that 
both  demonstrate  the  lack  of  direct  correspondence  between  depth  and  disparity,  and 
relative  insensitivity  to  constant  disparity  gradients.  But  the  analogy  has  limits:  while  the 
reconstructions  appear  to  embody  similar  computational  principles,  the  detection  of  the 
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underlying  contrast  or  discontinuity  events  in  the  two  domains  is  probably  achieved  by 
different  methods.  We  first  review  the  case  regarding  depth  reconstruction. 

4.1  Depth  reconstruction 

The  notion  that  stereo  depth  is  reconstructed  indirectly  from  disparity  contrast,  much  as  is 
brightness  from  luminance  contrast,  is  not  particularly  intuitive.  The  optical  geometry  of 
the  two  images  has  been  shown  by  many  theoretical  analyses  to  support  the  direct 
pointwise  computation  of  spatial  information  such  as  depth,  slant  and  absolute  distance, 
provided  that  the  necessary  optical  parameters  are  known  from  either  retinal  or  extraretinal 
sources  (Foley  1980;  Mayhew  and  Longuet-Higgins  1982;  Prazdny  1983).  For  simple 
binocular  arrangements,  often  a  pair  of  lines,  it  has  been  shown  that  the  perceptions  of 
depth,  relative  distance  and  absolute  distance  are  all  rather  accurately  predicted  by  the  direct 
geometric  relationships,  with  systematic  errors  that  can  be  attributed  to  misperception  of  the 
actual  angle  of  convergence,  differential  magnification  in  the  two  eyes,  and  so  forth  (see 
review  in  Foley  1980).  Moreover,  the  optical  geometry  suggests  that  apparent  depth 
should  vary  linearly  with  disparity  but  with  the  square  of  the  observation  distance.  This 
effect,  called  depth  constancy,  is  particularly  apparent  for  small  disparities  and  near 
observation  distances  (Ono  and  Comerford  1977;  Ritter  1979;  Wallach,  Gillam  and 
Cardillo  1979).  It  had  been  assumed,  more  or  less  tacitly,  that  such  results  would  also 
apply  to  a  continuous  binocular  surface,  e.g.  with  apparent  depth  varying  according  to  the 
disparity  at  each  surface  point.,  apparent  surface  slant  varying  according  to  the  disparity 
gradient,  and  so  forth. 

Despite  the  elegance  of  the  geometric  equations  and  their  predictions  under  certain 
controlled  experimental  circumstances,  other  observations  argue  against  a  direct  depth 
computation.  As  mentioned  earlier,  apparent  depth  remains  invariant  over  differing  retinal 
motions  in  the  two  eyes,  which  suggests  that  depth  derives  from  the  relative  arrangement  of 
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disparities,  and  not  their  absolute  retinal  coordinates  (Steinman  &  Collewijn  1980;  Lappin, 
1985).  Furthermore,  the  particular  spatial  arrangement  of  binocular  features  also  matters, 
as  demonstrated  by  depth  attraction  or  repulsion  between  adjacent  features  and  the 
diminished  depth  from  coplan ar  arrangements  of  binocular  features  (McKee  1983;  Gillam, 
Flagg  and  Finlay  1984;  Mitchison  and  Westheimer  1983;  Stevens  and  Brookes  1988). 
These  observations  together  suggest  an  indirect  relationship  between  disparity  and  depth 
for  disparity  distributions  associated  with  continuous  surfaces.  Rogers  (1986),  noting  that 
second  spatial  derivatives  of  disparity  are  invariant  over  changes  in  viewing  distance,  has 
proposed  that  surface  curvature  features  provide  the  basis  for  phenomenal  depth  constancy. 
In  general,  depth  across  continuous  surfaces  seems  to  derives  indirectly  from  surface 
curvature  features,  which  correspond  to  places  where  the  second  spatial  differences  of 
disparity  are  nonzero  (Stevens  and  Brookes  1987,  1988),  or  in  other  words,  where  a 
gradient  of  relative  disparities  exists  (Gillam,  et  al.  1988),  which  corresponds  to 
differences  of  first  differences  (Mitchison  and  Westheimer  1984). 

Thus  the  rather  direct  relationship  between  depth  and  disparity,  demonstrated  for 
isolated  3D  features,  does  not  apply  to  the  depth  across  continuous  surfaces.  In  particular, 
when  disparity  varies  linearly,  as  would  occur  in  viewing  a  continuous  slanted  plane, 
apparent  depth  is  determined  by  the  disparity  contrast  across  the  borders  of  the  plane 
relative  to  the  background,  if  available.  In  the  absence  of  border  disparity  contrast,  the 
slant  of  the  plane  in  depth  is  dominated  by  the  monocular  interpretation  (Stevens  and 
Brookes  1988). 

In  the  luminance  domain,  brightness  contrast  effects  reflect  limitations  in  the  visual 
system's  ability  to  reconstruct  a  luminance-related  signal  from  measures  of  luminance 
change,  presumably  by  interpolation  (e.g.  by  lateral  facilitation)  within  regions  bounded  by 
contrast  features  (Gerrits  and  Vendrick  1970;  Davidson  and  Whiteside  1971;  Arend  1973; 
Frisby  1979;  Arend  and  Goldstein  1987).  The  stereo  analogues  suggest  that  binocular 
depth  is  likewise  reconstructed,  i.e.  interpolated  within  regions  bounded  by  disparity 
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contrast  features.  While  the  exact  nature  of  the  disparity  features  is  not  well  understood, 
depth  is  elicited  most  effectively  where  the  second  spatial  differences  of  disparity  are 
nonzero,  which  correspond  to  surface  discontinuity  and  curvature  features  (Stevens  and 
Brookes  1987, 1988).  And  just  as  constant  luminance  gradients  are  effectively  featureless 
and  difficult  to  perceive,  constant  disparity  gradients  are  similarly  devoid  of  surface 
features  and  their  interpretation  in  depth  depends  largely  on  the  availability  of  disparity 
contrast,  e.g.  along  their  borders  (Gillam  et  al.  1984;  Stevens  and  Brookes  1987;  Gillam  et 
al.  1988;  Stevens  and  Brookes  1988). 

42  Discontinuity  detection,  spatial  differentiation  and  lateral  inhibition 

The  important  binocular  disparity  features  vis-a-vis  surface  reconstruction  appear  to 
correspond  to  loci  where  the  second  spatial  differences  of  disparity  are  nonzero.  Such 
features  would  be  detected  by  measuring  second  spatial  derivatives  of  disparity.  Spatial 
differentiation  can  be  achieved  effectively  by  center-surround  lateral  inhibition  operators,  a 
strategy  that  seems  general  to  sensory  processing.  While  in  the  luminance  domain  the 
differentiation  appears  to  be  achieved  by  a  circular-symmetric  Laplacian-like  filter,  the 
known  orientation  anisotropy  in  sensitivity  to  disparity  change  would  suggest  against  a 
circular- symmetric  operator  for  the  corresponding  detection  of  disparity  features.  Instead, 
one  might  postulate  directional  derivative  operators  composed  of  elongated  receptive  fields 
with  lateral  inhibition  between  adjacent  subfields. 

As  discussed,  there  is  evidence  for  the  existence  of  very  short  range  (several  arc 
minute)  spatial  lateral  facilitation  and  inhibition  in  stereopsis.  The  effective  spatial  MTF  of 
sensitivity  to  stereo  depth  also  suggests  lateral  inhibition.  But  when  one  examines  the 
stereo  analogous  to  the  traditional  Mach  band  stimuli  and  the  Hermann  grid,  the  expected 
lateral  inhibition  effects  are  not  readily  apparent.  We  see  three  alternative  explanations. 
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First,  the  lateral  inhibition  effects  in  depth  may  simply  be  more  subtle  than  our 
explorations  allowed  for,  or  were  masked  by  the  experimental  design.  But  if  the  measured 
MTF  for  stereopsis  is  taken  as  an  indication  of  the  size  of  the  underlying  receptive  fields, 
and  if  these  receptive  fields  are  presumed  to  spatially  summate  disparities  in  the 
conventional  lateral-inhibitory  manner,  their  effects  would  presumably  not  be  particularly 
subtle. 

The  second  alternative  is  suggested  by  the  conventional  wisdom  that  relative,  if  not 
absolute,  binocular  disparities  are  available  after  binocular  fusion.  Differentiation-like 
filtering  of  their  spatial  distribution  would  serve  to  detect  possible  surface  features 
(discontinuities  and  other  curvature  events).  As  in  luminance  processing,  the 
differentiation  operator  would  produce  patterns  of  activity  that  could  be  misinterpreted  (e.g. 
Mach  bands).  But  unlike  luminance  processing,  which  has  only  limited  access  to  the 
original  luminance  signal,  disparity  processing  could  independently  determine  from  the 
disparities  in  the  immediate  vicinity  of  each  possible  feature  true  features  from  artifacts. 
We  see  no  way  to  test  this  alternative  given  the  current  state  of  understanding,  nor  to 
distinguish  this  from  the  following  alternative. 

The  third  alternative  is  that  disparity-contrast  features  (edges  and  other  curvature- 
related  surface  properties)  are  detected  by  processes  that  do  not  induce  the  characteristic 
lateral  inhibition  effects  reported  by  others.  While  both  luminance  contrast  and  disparity 
contrast  features  seemingly  require  localizing  changes  in  gradient  (i.e.  non-zero  second 
spatial  differences),  they  are  unlikely  detected  by  analogous  operators.  It  would  be 
disadvantageous  to  perform  spatial  differentiation  by  disparity-sensitive  receptive  fields 
which,  by  analogy,  summate  all  disparity  signals  within  small  neighborhoods.  To  do  so 
would  be  to  blur  not  only  in  the  two  spatial  dimensions  of  the  image,  but  in  depth  as  well. 
This  would  pose  problems  for  the  perception  of  transparent  urfaces,  where  in  a  given 
visual  direction  at  least  two  surfaces  planes  of  disparities  might  be  expected.  It  would  be 
preferable  to  perceptually  segregate  distinct  those  disparity  signals  likely  associated  with 
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separate  surfaces,  prior  to  attempting  to  detecting  surface  features.  This  alternative  expects 
that  those  disparity  distributions  consistent  with  coherent  surfaces  (e.g.  as  measured  in 
terms  of  local  autocorrelation  of  disparity  or  local  coplanarity)  are  treated  differently  than 
incoherent,  or  volume-filling,  distributions  (see  evidence  in  Brookes  &  Stevens,  1989). 

We  should  note  that  an  alternative  method  for  computing  a  (directional)  second 
difference  is  to  perform  two  consecutive  first-differences.  The  initial  first-difference 
operation  might  be  a  consequence  of  compensating  for  uncontrolled  disjunctive  and 
conjunctive  eye  movements  by  shifting  or  remapping  images  (Anderson  and  Van  Essen 
1987).  As  a  result,  positional  information  would  be  known  only  relatively  (within  each 
monocular  image  and  between  left  and  right  images).  The  loss  of  absolute  position 
information,  analogous  to  the  loss  of  absolute  luminance  information,  causes  simultaneous 
contrast  effects  in  motion  perception  as  well  as  stereopsis  to  (Loomis  &  Nakayama  1973; 
Rogers  1986;  Bowns  and  Braddick  1986). 

If  another  first-difference  operation  were  performed  on  the  remapped  images,  the 
result  would  approximate  a  second  directional  derivative  of  the  (motion  or  disparity)  fields. 
Spatial  differentiation  might  therefore  be  achieved  by  shifting  rather  than  convolution  by 
lateral-inhibition  operators.  There  are,  however,  substantial  control  issues,  such  as 
determining  the  scale  or  locality  over  which  a  given  shift  is  performed,  and  in  spatially 
delimiting  the  application  of  a  given  shift 

Remapping  or  shifting  is  a  particularly  elegant  solution  to  the  problem  of 
compensating  for  a  spatially-uniform  error  of  unknown  magnitude,  where  the  relative 
signal  is  more  reliable  than  the  absolute.  Anderson  and  Van  Essen  (1987)  expect  the  shifter 
to  be  controlled  by  a  combination  of  feedforward  (e.g.  direct  estimation  of  the  local  signal 
to  nullify)  and  feedback  (e.g.  minimizing  residual  error  or  maximizing  measure  of 
registration)  strategies.  Furthermore,  if  the  magnitude  and  direction  of  the  shift  were 
determined  locally  for  sufficiently  small  regions,  the  effect  would  be  removing  or  reducing 
constant  gradients  as  well  as  spatially  uniform  terms.  Local  remapping  would  thus  account 
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for  insensitivity  to  low  spatial  frequency  disparity  changes,  as  characteristic  of 
differentiation  operators.  But  it  would  also  induce  the  depth  artifacts  in  the  vicinity  of 
disparity  discontinuities  that  are  also  characteristic  of  differentiation.  Furthermore,  the 
choice  of  control  strategy  is  particularly  difficult  for  small  populations  of  binocular 
features,  such  described  in  (Mitchison  and  Westheimer  1984,  figure  5).  While  remapping 
may  contribute  to  the  removal  of  low  spatial  frequency  disparity  information,  it  appears  that 
the  distribution  of  relative  disparities  is  explicitly  analyzed  for  planarity,  as  part  of  the 
extraction  of  surface  discontinuity  and  curvature  information. 

In  summary,  stereo  depth  and  brightness  are  analogous  in  that  both  are 
reconstructions.  Just  as  apparent  brightness  is  dominated  by  the  distribution  of  contrasts, 
stereo  depth  is  dominated  by  the  distribution  of  disparity  contrasts.  The  analogy  does  not 
extend,  however,  to  the  corresponding  contrast-detection  mechanisms. 
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1.  A  variant  of  the  Koffka  ring.  In  a  a  constant  luminance  zing  is  embedded  in  a  uniform 
luminance  gradient  background.  In  b  the  stereo  disparity  analogue  presents  a  constant 
disparity  ring  against  a  uniform  disparity  gradient  background.  Note  that  the  ring  appears 
slanted. 


2.  Similar  to  figure  1  but  with  a  background  with  Gaussian  profile.  In  a  manner  analogous 
to  the  variable  brightness  seen  in  the  constant-luminance  ring  in  a,  the  constant-disparity 
ring  in  b  appears  curved  in  depth. 


tereo  analogue  to  the  simultaneous  contrast  effect.  The  stairsteps  appear  slant) 


5.  Stereo  analogue  to  the  Hermann  Grid. 


